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(54) Methods of and apparatus for hands-free operation of a voice recognition system 



(57) Techniques for activating a voice recognition 
system are described. The system disclosed can be 
non verbally started and stopped without the need for 
the system user to use his or her hands facilitating 
hands-free operation in an environment with a relatively 
high degree of ambient background verk>al noise. For 
example, a user may start the voice recognition system 
by blowing once on a microphone and stop the system 
by blowing twice on a microphone. The user may alter- 
nately start and stop the system by using other nonver- 
bal inputs such as facial expressions, clearing the throat 
or moving one's head. Such other nonverbal inputs may 
be detected by pressure sensors or motion sensors. 
The operational state of the system may alternately be 
toggled from off to on, or from on to off, with each sub- 
sequent nonverbal input. 
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Description 

[0001] The present invention relates generally to 
voice recognition systems, and in particular to improve- 
ments in the use of such systems in an environment s 
where hands-free operation is desirable, such as a retail 
checkout station. 

[0002] The invention relates to advantageous meth- 
ods of turning a voice recognition system on and off 
without the need for the system operator to use their io 
hands to push a button or click a mouse, or to issue spe- 
cific voice commands. The methods and apparatus 
desaibed in further detail below are well suited to noisy 
environments where a traditional voice recognition sys- 
tem would have difficulty ascertaining the operator's is 
voice from background noise, and prevent the system 
from being accidentally activated by a person other than 
the operator. 

[0003] Retailers have traditionally used a variety of 
methods to automate the retail customer transaction 20 
process. The desire to automate this process is driven 
by the competing goals of completing the transaction 
quickly and accurately, while providing the customer 
with a positive shopping experience and gathering 
increasing amounts of information about the transaction 25 
and the customer. 

[0004] Retail grocers have been the most progres- 
sive type of retailer in regard to automating the checkout 
process. The near universal implementation of product 
package t»ar-coding by product manufacturers, and bar- 30 
code scanning by retailers, has been essential to the 
implementation of the automation process. One signifi- 
cant problem that remains is the impact that unlabeled 
products, or otiienwise un-scannable products, have on 
the automation of the checkout process. This problem is 35 
further compounded by the lack of training and relatively 
low skill level of cashiers due to rapid employee turnover 
and other factors. Among the most common reasons for 
an item being unscannable are that the barcode is dam- 
aged or of insufficient quality, the item is not found in the 40 
FOB system price look-up (PLU) file, or the product 
package is notbarcoded. When a cashier encounters a 
product that is unscannable, the cashier must typically 
stop the scanning process and manually take some 
action to identify the item. If the item has a barcode, but 45 
the item does not scan, the operator rnay try typing the 
Universal Product Code (UPC) number on the package. 
If the item consists of unlabeled bulk goods or produce, 
the operator may consult a paper list to find the proper 
price look-up (PLU) number. If neither of the above so 
actions is adequate or appropriate, the operator may 
call for assistance from another employee, enter a ficti- 
tious price or just give the item away 
[0005] The most common type of non-barcoded 
product in a grocery store is produce. Although many 55 
grocers have implemented tiie practice of labeling all 
produce with a PLU number printed on a sticker, individ- 
ual pieces of produce often lose the sticker by the time 
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the item reaches the cashier. The cashier will either 
enter a PLU from memory or consult a table of PLU 
numbers. It is common for cashiers to enter a PLU for a 
different, but similar, product merely to keep the check- 
out process moving quickly This deprives the grocer of 
accurate sales and inventory replenishment data. 
Another common scanning problem occurs when the 
POS system does not find the item in the PLU file. In 
this case the cashier may ask the customer for the item 
price, or call for assistance from a supervisor or grocery 
bagger, resulting in a significant delay in completing the 
checkout process. If the customer provides the price, 
the retailer may be deprived of accurate data, and pos- 
sibly revenue. If the cashier calls for assistance, cus- 
tomer satisfaction suffers while all of the customers in 
the line endure a delay Lastly if the barcode is dam- 
aged or otherwise unscannable, the cashier must man- 
ually enter the UPC number. If the UPC number is found 
in the PLU file, the checkout process proceeds with a 
nominal delay If the item is not found, the cashier must 
call for assistance and the customer is delayed further. 
A cashier experiencing any of the above situations 
could utilize a voice recognition system to query the 
POS system for a PLU inquiry or to voice certain prod- 
uct name or description information to be used in a 
search query The problem to be solved, then, is to be 
able to activate the voice recognition system only when 
needed, and to do so while continuing to scan other 
items. 

[0006] A similar set of problems are present in the 
food preparation industry This includes, but is not lim- 
ited to, the kitchens of a fine-dining restaurant, fast-food 
restaurant, or grocery store deli counter. Cooks and 
order takers in such an environment must use their 
hands to prepare meals or orders, and cannot work effi- 
dentiy if they must stop their work to push a button or 
dick a mouse to activate a voice recognition system. 
One major challenge for the kitchen of a fine-dining res- 
taurant is to be able to complete all orders for a given 
table at the same time, and to notify the appropriate 
waiters that their orders are ready to be taken to the 
tables. 

[0007] In a typical fast-food restaurant, the order 
takers enter orders into an order-entry computer sys- 
tem. The order- entry computer system then sends the 
kitchen portion of the order to a computer display in the 
kitchen. The cooks then remove orders from the kitchen 
display after they prepare the kitchen orders and 'bump* 
those orders back to the order fillers. The order fillers or 
order takers gather the non-kitchen portions of the order 
while the kitchen prepares the kitchen portion. When 
the kitchen items are bumped' back to the order fillers, 
the order fillers take the kitchen portion of the order and 
combine it with the non-kitchen portion. When the order 
is completely filled, the order filler delivers the order to 
the customer and "bumps' the order out of the order- 
entry system. The bumps' are typically accomplished 
by using a bump-bar peripheral device attached to the 
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order-entry system. Utilizing a typical bump-bar 
requires the order filler or cook to walk over to where the 
bump-bar is installed and physically push the bump-bar. 
[0008] The clerks at a grocery store deli counter 
need to weigh or measure orders before printing a label 5 
that identifies and prices those orders. A typical counter 
will have a plurality of scales that may be used by any 
one of the clerks. A clerk typically needs to perform a 
number of tasks to complete an order. These tasks may 
include filling small containers from large bulk contain- 
ers in a refrigerated case, cutting a loaf of bread on a 
bread slicer, and cutting meat products on a meat slicer. 
After each individual order item is prepared, the clerk 
must go to one of the scales to weigh the item and enter 
some identifying inforrmtion before the scale can gen- 
erate an appropriate label. 

[0009] In each of the above situations, the clerks, 
cooks and order takers could advantageously utilize a 
voice recognition system to provide the required bumps 
or data inputs. Utilizing a voice recognition system that 
is activated in a hands-free manner would allow those 
workers to provide the necessary data inputs while they 
were completing their other required tasks. 
[0010] The present invention recognizes that there 
exists a need in a variety of contexts for a hands-free 
method of starting and stopping a voice recognition sys- 
tem. Such a method shouki advantageously allow the 
starting and stopping of such a system without the use 
of specific spoken conmands. Such a method should 
additionally operate Irrespective of the type of ambient 
noise present where such a system is installed. 
[001 1 ] Methods and apparatus for operating a voice 
recognition system in a retail store, in accordance with 
one aspect of the present invention, preferably include a 
means of starting and stopping the recognition process 
without the use of the operator's hands or feet, as well 
as a means for annunciating the status of the voice rec- 
ognition process. One method described further below 
allows for starting and stopping the voice recognition 
system by blowing on a microphone. Alternate methods 
described below allow an operator to start and slop the 
voice recognition system by coughing, clearing the 
throat, moving the head or other parts of the body in 
one embodiment of the present invention, the system 
may be started or stopped by blowing, coughing, or the 
like, once or twice respectively Alternately, the state of 
the voice recognition process may be toggled between 
the 'started' and 'stopped' states by blowing, coughing, 
or the like, into a microphone or pressure sensor. 
[0012] Embodiments of the present invention will 
now be described, by way of example, with reference to 
the accompanying drawings, in which: 
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Rgure 2 illustrates a headset and a headset 
adapter for inplementing one embodiment of the 
present invention; 

Rgure 2a illustrates an alternate headset and 
headset adapter for implementing another embodi- 
ment of the present invention; and 

Rgure 3 illustrates a process for monitoring a 
microphone and determining when to switch a 
voice recognition system on or off in accordance 
with the present invention. 

[0013] Rgure 1 illustrates a POS terminal 100 
adapted for use in conjunction witii the present inven- 
tion as described further below. A base unit 110 
includes a microprocessor 111, memory 112 and a 
communications adapter 113. The POS terminal 100 
communicates with other computers via the communi- 
cations adapter 1 13 via a local area network connection 
150. Peripheral input and output devices are connected 
to the base unit 110 including an operator keyboard 
121, an operator display 123, a cash drawer 120, a 
receipt printer 122 and an annunciator 1 U. Annunciator 
114 may be a beeper, speaker, lighted indicator, or 
some combination thereof, and may alternately be 
incorporated into base unit 110. An operator headset 
130 is also connected to the base unit. 
[0014] The POS terminal 100 is adapted for use in 
conjunction with the present invention by utilizing the 
microprocessor 1 1 1 in conjunctbn with a software pro- 
gram stored in the memory 1 12 to analyze Irput signals 
from the operator headset 130. 
[0015] The connection between the operator head- 
set 130 and the POS terminal base unit 110 may be a 
direct wired connection as illusti'ated in Rgure 1. Atypi- 
cal alternate connection between the operator headset 
130 and the POS terminal base unit 1 10 is illusti'ated in 
Figure la. TTie operator headset 130 is connected to 
the wireless transmitter/receiver 151 by a wired connec- 
tion. The wireless transmitter/receiver 1 52 is connected 
to the POS terminal base unit 1 10 by a wired connec- 
tion. The wireless transmitter/receiver 151 communi- 
cates with the wireless transmitter/receiver 152 via a 
radio frequency, infrared, or other common wireless 
communication linK allowing a wearer of the headset 
130 to move freely within the range of the link.. 
[0016] Rgure 2 illustrates an apparatus 200 for use 
in implementing the present invention. An operator 
headset 21 0 is connected via connection 21 5 to a head- 
set adapter 220 installed in a computer 230 such as the 
base unit 1 10 of the POS terminal 100 as illustrated in 
Figure 1. The connection 215 may be a direct wired 
connection as Illustrated in Rgure 1 or it may include a 
wireless communication link as illustrated In Rgure la. 
The operator headset 210 minimally consists of a micro- 
phone 21 1 and may include one or more speakers 212. 
The connection 215 allows audio signals to pass from 
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Figure 1 Illustrates a POS terminal suitable for use 
in conjunction with the present invention; 55 

Figure la illustrates an alternative POS terminal for 
use In conjunction with the present invention; 
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microphone 211 to the headset adapter 220. and for 
audio signals to pass from the headset adapter 220 to 
the optional speaker or speakers 212. The headset 
adapter 220 includes filter 221 and start/stop logic 222. 
[001 7] Tlie filter 22 1 is designed with a passband to 5 
pass only those frequendes that represent the nonver- 
bal start and stop signals. These start and stop signals 
are passed to start/stop logic 222, which in turn signal 
the voice recognition system software to respectively 
start and stop accepting audio inputs from operator io 
headset 210. In one embodiment of the present inven- 
tion, filter 221 is designed to pass certain frequencies 
that represent nonverbal sounds, such as blowing, 
coughing, clucking or the like. RIter 221 may include a 
sampling function to be utilized in a training mode 75 
wherein background ambient noise may be sampled 
and measured, a minimum and maximum input thresh- 
old level may be determined, and a frequency map for 
the desired start and stop sounds, and the like, for a 
given operator, may be stored. This training mode may 20 
be initiated each time an operator uses the system or it 
may be Initiated once for each of the operators that may 
use the system. This training mode may alternately be 
accomplished by the voice recognition system software, 
and the appropriate parameters then utilized to initialize 25 
filter 221. While Figure 2 illustrates a separate hardware 
filter, a software implementation may also be employed. 
[0018] Figure 2a Illustrates an apparatus 250 for 
use in Implementing an alternate embodiment of the 
present invention. An operator headset 260 Is con- 30 
nected via connection 265 to a headset adapter 270 
installed in a computer 280 such as the base unit 1 10 of 
the POS terminal 100 as illustrated in Figure 1. The 
connection 265 may be a direct wired connection as 
illustrated in Figure 1 or it may include a wireless com- 35 
munication link as illustrated in Figure la. The operator 
headset 260 minimally consists of a microphone 261, a 
pressure sensor 263 and may include one or more 
speakers 262. The connection 265 allows audio signals 
to pass from microphone 261 to the headset adapter 40 
270, and for audio signals to pass from the headset 
adapter 270 to the optional speaker or speakers 262. 
The headset adapter 270 separates tiie audio signals 
from microphone 261 from the start and stop command 
signals from pressure sensor 263. These start and stop 45 
signals are passed to the voice recognition system soft- 
ware, commanding the voice recognition system to 
respectively start and stop accepting audio inputs from 
operator headset 260. 

[001 9] In one embodiment of the present invention, so 
pressure sensor 263 is collocated with microphone 261 . 
Pressure sensor 263 is designed to. detect the vocal 
pressure corresponding to a known, nonverbal start or 
stop command. Pressure sensor 263 may include a 
sampling function to be utilized in a training mode 55 
wherein background ambient noise sound pressure 
may be sampled and measured, and a minimum and 
maximum input threshold level may be determined and 



stored. This training mode may be initiated each time an 
operator uses the system or It may be initiated once for 
each of the operators that may use the system. This 
training mode may alternately be accomplished by the 
voice recognition system software, and the appropriate 
parameters then utilized in conjunction with pressure 
sensor 263. 

[0020] In another embodiment of tiie present inven- 
tion, pressure sensor 263 is designed to detect facial 
movements or expressions such as moving the jaw, 
raising an eyebrow or pressing a cheek against the sen- 
sor. In this embodiment, pressure sensor 263 is not col- 
located with microphone 261. Instead, pressure sensor 
263 Is incorporated into operator headset 260 in such a 
way as to allow it to be placed in contact with the opera- 
tor's jaw, cheek, or the like. 

[0021] Figure 3 illustrates a procedure 300 for 
determining when to switch a voice recognition system 
on and off in accordance with one aspect of the present 
invention. The process begins at process step 302 with 
the voice recognition system switched off. At step 304, 
the process monitors and analyzes system input sig- 
nals. At step 306, it is determined rt the input signal 
matches the criteria for switching the voice recognition 
system on. If the input signal does not match the criteria 
for switching the voice recognition system on. the proc- 
ess loops back to step 304. If the input signal does 
match the criteria for switching tiie voice recognition 
system on, the process proceeds to step 308 and the 
voice recognition system is switched on. The process 
tiien proceeds to step 310 where the process monitors 
and analyzes further system input signals. At step 312, 
the process determines if the input signal matches the 
criteria for switching the voice recognition system off. If 
the Input signal does not match tiie criteria for switching 
the voice recognition system off, the process loops back 
to step 308. If the input signal does match the criteria for 
switching the voice recognition system off, the process 
proceeds to loop t>ack to step 302 and the voice recog- 
nition system is switched off. 

Claims 

1. A hands-free method for starting and stopping a 
voice recognition system, the method comprising 
the steps of: 

storing in a computer memory a set of refer- 
ence nonverbal voice signals, said voice sig- 
nals representing commands to start and stop 
a voice recognition system; 

reading into a computer memory a nonvert>al 
voice signal from a microphone; 

comparing said voice signal from said micro- 
phone to said set of reference voice signals; 
and 
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sending start or stop commands to a voice rec- 
ognition system if said voice signal from said 
microphone matches one of the set of said ref- 
erence voice signals. 

5 

A method as claimed in claim 1 , wherein said set of 
reference voice signals includes; 

a nonverbal voice signal to be provided a first 
predetermined number of times, which gener- io 
ates a command to start the voice recognition 
system; and a nonverbal voice signal to be pro- 
vided a second predetermined number of 
times, which generates a command to stop a 
voice recognition system, said first and second is 
predetermined numbers being different. 

A method as claimed in claim 2. wherein said first 
predetermined is one and said second predeter- 
mined number is two. 20 

A method as claimed in any preceding claim, 
wherein said reference non-verbal signal com- 
prises one of the following: blowing into the micro- 
phone, clearing the throat, ducking the tongue or 25 
moving the head or other parts of the body. 

A hands-free method for starting and stopping a 
voice recognition system, the method comprising 
the steps of: so 

storing in a computer memory a reference non- 
vert)al voice signal, said voice signal represent- 
ing a corfimand to toggle the operational state 
of a voice recognition system; 35 

reading into a computer memory a nonverbal 
voice signal from a microphone; 

comparing said voice signal from said micro- 40 
phone to said reference voice signal; and 

sending a toggle command to a voice recogni- 
tion system if said voice signal from said micro- 
phone matches said reference voice signal, 45 
said toggle command switching the voice rec- 
ognition system on if it is off or off if it is on. 

A method as claimed in claim 5, wherein said refer- 
ence voice signal is comprised of a nonverbal voice so 
signal taken from the following: representing the 
souixl of blowing into the microphone; clearing the 
throat or ducking the tongue. 

An apparatus for non verbally starting arxl stopping 55 
the operation of a voice recognition system, said 
apparatus comprising: a microphone; a filter; and a 
logic circuit for controlling the operation of a voice 



recognition system, said logic circuit being operable 
to monitor the output of said filler and being further 
operable to determine when to issue start and stop 
commands to said voice recognition system based 
upon the output of the filter exceeding a predeter- 
mined threshold level. 

8. An apparatus as claimed in claim 7, 8 comprising 
an annunciator, said annunciator indicating the 
operational status of the voice recognition system. 

9. An apparatus as claimed in claim 7 or 8. 8 compris- 
ing a headset, said headset comprised of said 
microphone and one or more speakers. 

10. An apparatus as claimed in any of claims 7 to 9, 
wherein said filter and said logic circuit are con- 
tained on an adapter card for use with a POS termi- 
nal. 

11. The apparatjs of claim 10, wherein said filter is an 
analog-to<!igital converter. 

12. A method of calibrating an apparatus for controlling 
the operation of a voice recognition system, the 
method comprising the steps of: 

measuring the volume and frequency content 
of the ambient bacJ^ground noise from a micro- 
phone to generate a first calibration signal; 

measuring the volume and frequency of non- 
vertjal voice signals from a microphone to gen- 
erate second and third calibration signals, sakJ 
voice signals representing start and stop com- 
mands for use with the voice recognition sys- 
tem; and 

modifying the volume and frequency response 
of said filter to correspond to the first, second 
and third calibration signals. 

13. A method as claimed in claim 12, comprising tiie 
steps of: 

determining minimum and maximum threshold 
volume levels by voicing nonverbal test signals 
into said microphone. 

14. A method of calibrating an apparatus for controlling 
the operation of a voice recognition system, the 
method comprising the steps of: 

measuring the volume and frequency content 
of the ambient background noise from a micro- 
phone to generate a first calibration signal; 

measuring the volume and frequency of a non- 
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verbal voice signal from a microphone to gen- 
erate a second calibration signal, said voice 
signal representing a toggle command for use 
with the voice recognition system; and 

modifying the volume and frequency response 
of said filter to correspond to the first and sec- 
ond calibration signals. 
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